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An improved measurement of the flux distribution of the 
Lya forest in QSO absorption spectra: the eff^ect of 
continuum fltting, metal contamination and noise 
properties'^ 



1^ 

o 
o 

> 

o 
m 



6 



> 

(N 

oo 



o 



X 



T.-S. Kimi'2, J. S. Bolton2'3, M. Viel^'^, M. G. Haehnelt^, R. F. Carswell^ 

^ Astrophysikalisches Institut Potsdam, An der Sternwarte 16, D-14482 Potsdam, Germ,any 

^ Institute of Astronomy, Madingley Road, Cambridge CBS OH A, UK 

^ Max- Planck- Institut fiir Astrophysik, Karl-Schwarzschild-Str. 1, 8574-8 Garching bei Miinchen, Germany 

^ INAF-Osservatorio Astronomico di Trieste, Via G. B. Tiepolo 11, 1-34131 Trieste, Italy 



Received 2007 July 7; Accepted 2007 September 3 



ABSTRACT 

We have performed an extensive Voigt profile analysis of the neutral hydrogen (H i) 
and metal absorption present in a sample of 18 high resolution, high signal-to-noise 
QSO spectra observed with VLT/UVES. We use this analysis to separate the metal 
contribution from the H I absorption and present an improved measurement of the flux 
probability distribution function (PDF) due to Hi absorption alone at <z>= 2.07, 
2.52, and 2.94. The flux PDF is sensitive to the continuum fit in the normalised flux 
range 0.8 < F < 1.0 and to metal absorption at 0.2 < F < 0.8. Our new measurements 
of the flux PDF due to Hi absorption alone are systematicall y lower at 0.2 < F < 0. 8 
by up to 30% compared to the widely used measurement of iMcDonald et all (J2000D . 
based on a significantly smaller sample of Keck/HIRES data. This discrepancy is 
probably due to a combination of our improved removal of the metal absorption and 
cosmic variance, since variations in the flux PDF between different lines-of-sight are 
large. The Hi effective optical depth r^*[ at 1.7 < z < 4 is best fit with a single 
power law, t^^ = (0.0023 ± 0.0007) (1 + z)^'^'^^"^!, in good agreement with previous 
measurements from comparable data. As also found previously, the effect of noise on 
the flux distribution is not significant in high resolution, high signal-to-noise data. 
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1 INTRODUCTION 

The Lya forest refers to the numerous, narrow absorp- 
tion features bluewards of the Lya emission line in the 
spectra of high-redshift QSOs. Our understanding of the 
origin of the Lya forest has made major p rogress with 



the advent of high r e solution spectro s copy llCowie et al 
iLu et all Il996l: 



19951 : iHu et al.l [19951: 
19971: lRauchetal.1 1 19971: 



Kirkman fc Tvtler 



20001 : 



_ ■ ■ ■ ■ .Cristiani fc D'Odoricd . 

Kim et al. II2001I : Ijanknecht et al.ll2006l ) and the availability 
of hy drodynamic cosmolog ical sim ulations ijCen et al.l 



19941: iMirald a-Escud e et al.! 1 19961: IZhang et al 



Weinberg et al.,,1998, : .Croft et al. II 19981 : iTheuns et all 



199 



199. 



* Based on data taken from ESO archive obtained with UVES 
at VLT, Paranal, Chile. 



Bryan et al 



Jena et aU 



19991 : iDave et all 1 19991 : ICroft et al. I I2OO2I : 
2005|). Comparison of the predictions from 



hydrodynamic cosmological simulations with observations 
have established a picture where the Lya forest is due to the 
neutral hydrogen (Hi) component of the warm (~ lO^K) 
photoionised intergalactic medium (IGM) which traces 
moderate- amplitude density fluctuations of the (dark) 
matter distribution in a simple manner. In this picture 
the spectra of high-2: QSOs provide a one-dimensional 
probe of the matter density along the line-of-sight to high 
redshift QSOs. The flux distribution in QSO absorption 
spectra thus encodes information on the underlying matter 
distribution and evolution and the Lya forest has been 
recognised as a powerful cosmological tool, complementing 
other cosmological probes. 

So far it is mainly the power spectrum of the flux dis- 
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tribution which has been used to quantitatively probe the 
properties of the large scale distribution of matter and its de- 
pendence on cosmological parameters. The flux power spec- 
trum is sensitive to the matter power spectrum over a wide 
redshift range (2 < z < 4) and on scales of l-50/i~^Mpc, 
significantly smaller than those probed by CMB fluctua- 
tions, galaxy surveys and gravitational lensing (Croft et al. 
2002; Seljak, McDonald & Makarov 2003; Viel, Haehnelt 
& Springer 2004b; McDonald et al. 2006; Viel, Haehnelt & 
Lewis 2006; Lesgourges et al. 2007). 

The probability distribution of the flux is also 
sensitive to the spatial distribution of dark mat- 
ter and cosmological parameters, in particula r, the 
amplitude of matte r fluctuations ([Rauch et al.l 
Weinberg et all 1 19981: iTheuns. Schave fc Haehnelt! 



1997 



200C 



Meiksin. Bryan fc MachacekluOOlf) . However, the effect of 
the thermal state of the IGM (Theuns et al. 2000) and 
other uncertainties make it more difficult to extract this 
information from the flux PDF. First attempts at a joint 
analysis of t he flux PDF and the flux po w er spectrum have 
been made | Dcsiacques fc Nusser) l2005l : iLidz et al] l2006l : 
iDesiacgues. N usser fc Shcth 2007|). The analysis of Des- 
jaques et al. gives a somewhat smaller amplitude for the 
matter power spectrum compared to studies based on the 
flux power spectrum alone. Note, however, that these re- 
sults were based on dark-matter-only numerical simulations 
in which the gas distribution was modelled in an approxi- 
mate way rather than on full hydrodynamical simulations. 

The measured flux PDF is sensitive not only to 
the Hi absorption, but also, unfortunately, to continuum 
level uncertainties, contaminating heavy element absorp- 
tion, and noise. The effect of continuum fitting uncertain- 
ties on t he Hi effective optical depth has been discussed 
in dctaU ("Press. Rvbicki fc Schneidcr"l993':'Ki m et al.ll2002l: 
Bcrnardi ct al. 2003; Tytlor ct al. 2004; Kirkman et"aLl 
20051 '). but their impact on the flux PDF has not been con- 
sidered as extensively (see Rauch et al. 1997 and McDonald 
et al. 2000 for notable exceptions). Very little, if anything, 
has been done to quantify the effect of metal absorption 
and noise on the flux PDF. To remedy these omissions, we 
perform a detailed Voigt profile analysis of the metal and 
Hi absorption in 18 high resolution, high signal-to-noise 
(S/N) VLT/UVES QSO absorption spectra to assess the im- 
pact of continuum fitting uncertainties, metal absorption, 
and noise properties on the H I flux probability distribu- 
tion at 1.7 < z < 3.2 and the Hi effective optical depth at 
1.7 <z < 4. 

Our study is based on a factor of two larger sample 
and probes towards low er z than similar work based on 8 
Keck/HIRES spectra bv lMcDonald et all HqoO). It also dif- 
fers significantly in the way we perform the continuum fitting 
and, more importantly, in the level of the characterisation 
of metal absorption. Most previous estimates of the effective 
optical depth and the flux PDF due to H I absorption have ei- 
ther removed the me tal lines statistically (jTytler et al.ll2004l : 
iKirkman et al.l 120051 ). e xcluded the metal - contaminated re- 
gions from the study l|McDonald et al.l l2000l : iLidz et al 



20041 ) 



2006 ) or included absorption due to metal lines (|Songaila 
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We have carefully identified and fitted metal and H I ab- 
sorption lines with Voigt proflles for our sample of 18 spec- 
tra. The line fltting is essentially complete in the Lya for- 



Figure 1. The redshift range of the Lya forest in the spectra 
used for this study. The sample is divided into redshift bins. The 
thin dotted Hnes, thick solid lines and dashed Hnes represent the 
< z > = 2.07, < z > = 2.52 and <z>= 2.94 sub-sample, respec- 
tively. 



est region and redwards of the Lya emission lines, but less 
complete in the higher-order Lyman forest where the corre- 
sponding spectral regions were not always available. During 
the fitting process, we have adjusted the initially estimated 
continuum level by a localised continuum fit in order to get 
an overall satisfactory fit to multiple-transition metal lines 
and higher-order Lyman series lines. This approach enables 
us to remove the metal contribution in the forest directly and 
to recover one long continuous metal-free forest spectrum, 
and also to obtain an improved estimate of the contirmum 
level. 

The paper is organised as follows. In Section O we de- 
scribe the data used for our analysis. In Section|3l we present 
the Voigt profile fitting analysis used to remove the metal 
contribution in the forest. We investigate systematics of un- 
certainties affecting the flux PDF in Section|4] The observed 
flux PDF function and effective optical depth are presented 
in Section [5] Our conclusions are given in Section |6l 



2 THE DATA 

2.1 Description of the sample 

We have selected a sampl e of 18 spectra from the LUQAS 
sample IIKim et al.l 2004h of 2 7 spectra obtained with 
VLT/UVES (|Dekker et aLll2000l ). The spectra in LUQAS 
(Large sample of UVES QSO Absorption Spectra) were 
taken from the ESO archive and a large fraction of them 
were observed as pa rt of the Large Program 166.A-0106 
iBergeron et al.ll2004l ). The observations were performed in 
the period 1999-2004. All spectra have high signal-to-noise 
(S/N ^ 25-30) in the Lya forest region and a resolution of 
_R ~ 45 000. The wavelengths are helio-centric and vacuum- 
corrected. The spectra were sampled with pixels of width 
0.05 A. 

From the 27 spectra in the LUQAS sample, we have cho- 
sen those with a S/N in the forest region of at least 30-50, 
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a full coverage of the Lya forest and at least some coverage 
of the Ly/3 forest. Most of the 18 spectra cover the wave- 
length range of the higher order Lyman forest down to the 
atmospheric cut-off at 3050 A. Some, however, cover only a 
part of the Ly/3 forest due to their low emission redshift. For 
Lya absorption at z ^ 1.99 the corresponding Ly/3 absorp- 
tion is not covered in these optical spectra. Note that we 
did not include the spectrum of HE1341 — 1020 which would 
have met these criteria but is a mini-BAL. 

In Table [T] we list the characteristic properties of the 
18 QSO spectra in our sample. The signal-to-noise ratio is 
per pixel with the first (second) number corresponding to 
the lower (upper) limit of the wavelength range analysed. 
Note that the S/N is not constant across the spectrum. To 
avoid the proximity effect, we excluded a region correspond- 
ing to 4,000 km s~^ bluewards of the Lya emission line. 
Lyman limit absorption systems (LLSs) with column densi- 
ties lO^^'^cm"^ Sj A^Hi < lO^^cm"'^ are included in our flux 
distribution study, while 50 A segments of the spectra have 
been excised to the left and right of the centre of sub-damped 
Lya absorption systems (sub-DLAs, A''hi = 10^^~^^"^cm~^). 
None of our spectra contain a damped Lya system {Nn i ^ 
lO'^^'^cm"^) in the observed wavelength range down to ei- 
ther the atmospheric cutoff or at the Lyman limit disconti- 
nuity caused by LLSs or sub-DLAs along the spectra with 
non-zero observed fluxes. 

The Lya forest regions of the spectra were grouped into 
3 redshift bins with median redshifts <z> — 2.07, 2.52 and 
2.94, as shown in Fig.[T]and listed in Table Ell in Appendix 
A. We have chosen these redshift bins as a compromise be- 
tween minimising the Poisson fluctuations of the statistical 
quantities we are interested in and being able to investigate 
their redshift evolution. As we will show in section [STT] spa- 
tial fluctuations in the flux PDF are large and substantial 
redshift paths are necessary to reduce the the Poisson noise 
from individual absorption systems. If an object contains an 
absorption profile which straddles a redshift bin boundary 
we have assigned it to the redshift bin in which most of it 
lies, and adjusted the wavelength range contributed by that 
QSO accordingly. 

T he spectra used he re are essentially the same as those 
used in lKim et al.l (|200J) in the analysis of the LUQAS sam- 
ple. There are, however, three differences between the spec- 
tra in the LUQAS sample and the spectra analysed here: 

First, additional data for HE1122-1648, HE2347-4342 
and PKS2126— 158 has been obtained since the LUQAS data 
reduction. The additional da ta has been redu ced with MI- 
DAS/UVES as described in iKim et all (|2004|), and added 
using the UVES.popler (UVES POst PipeLine Echelle Re- 
duction) progran^j, resulting in significantly larger S/N for 
the spectra of these three QSOs. 

Second, we have re-reduced and performed a first-order 
fiux calibration for the spectra of four QSOs (Q0420— 388, 
HE0940-1050, Q0002-422 and HE2217-2818) in order to 
estimate the normalisation uncertainty between calibrated 
spectra and un-calibrated spectra. The UVES spectra do not 
normally have an absolute flux calibration, due to the lack 
of well-calibrated standard stars with spectra taken at the 



^ http://astronomy.swin.edu.au/inmurphy/UVES_popIer.htmI, 
kindly provided by Dr. M. Murphy. 



resolution of the instrumenlo. Consequently, most spectra 
in the LUQAS sample were not flux-calibrated. Even in the 
few cases where flux calibrations were performed, such as for 
HE2217— 2818, these were done echelle order by echelle order 
and are far from perfect. It is not uncommon for a standard 
star to have strong hydrogen Balmer series with a width 
larger than a echelle order (~ 25 A), and then the spectrum 
cannot be properly flux-calibrated in this way. A lack of a 
proper flux calibration and deficiencies in the flat-fielding 
procedure sometimes leave unwanted, oscillatory features in 
the extracted spectrum, with periods corresponding to the 
scale of one echelle order. The flux is then higher at the 
central wavelength and lower at both ends of a given echelle 
order compared to the flux at the same wavelengths from ad- 
jacent echelle orders. In addition, the non-calibrated spectra 
show a different contiimum shape for the same wavelength 
regions covered by the different CCDs. These deficiencies 
aggravate the difficulties of continuum fitting the spectra, 
especially at high redshifts. Instead of an order-by-order 
fiux calibration, we performed a first-order flux calibration 
as follows. We corrected each merged science spectrum us- 
ing the master response functior|j provided by ESO for a 
given observing period. When the master response functions 
were not available for a particular period, we scaled the un- 
calibrated spectra to the flux-calibrated spectra from other 
periods. Th e flux-calibrated s pectra were then combined as 
described in lKim et all l|2004l '). 

Third, and most importantly, we performed new contin- 
uum fits to all spectra in the sample optimised for a Voigt 
profile analysis. The normalised LUQAS spectra and the 
normalised spectra analysed here therefore differ slightly. 
Details of the continuum fitting are described in the next 
section. 



2.2 Continuum fitting 

The absorption due to intervening neutral hydrogen and 
metals is imprinted on the QSO emission spectrum, which 
has several, normally broad, emission lines overlaid on 
continuum emission from thermal and non-thermal radia- 
tion processes. The spectral energy distribution (SED) of 
the emission therefore varies significantly between different 
QSOs. 

Most statistical analysis of the absorption thus require 
the fitting of a continuum characterising the SED of the un- 
absorbed emission of the QSO (see Li dz et al. (2006) for a 
recent study that uses spectra that are not normalised in 
this way). Careful continuum fitting is particularly impor- 
tant for a Voigt profile analysis of QSO absorption spectra 
such as the one performed here. Estimating the unabsorbed 
emission is particularly difficult at high redshifts where an 
increasing fraction of the spectrum shows significant absorp- 
tion. The problem of contin uum fitting has been extensively 
discussed in the literature (|Kim et al.ll2002l : iBernardi et al.l 



^ The UVES pipeline reduction program determines the large- 
scale continuum shape from the ratio of the spectral shape of the 
target and the flat-field flux. For a fixed instrumental setting, the 
extracted spectra of the same object show a more or less similar 
shape of the large-scale continuum. 
^ http://www.eso.org/observing/dfo/quality/UVES/qc/std_qcl.html 
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Table 1. Analysed QSOs 



QSO 



■^LycK 



ALya(A) S/N Continuum (%)^ Lyman limit "^ notes'^ 



Q0055-269 


3.655<= 


2.936-3.205 


4785-5112 


80-50 


PKS2126-158 


3.279 


2.815-3.205 


4638-5112 


50-200 


Q0420-388 


3.116<= 


2.480-3.038 


4231-4909 


100-140 


HE0940-1050 


3.078 


2.452-3.006 


4197-4870 


50-130 


HE2347-4342f 


2.874= 


2.336-2.819 


4055-4643 


100-160 


Q0002-422 


2.767 


2.209-2.705 


3901-4504 


60-70 


PKS0329-255 


2.704-= 


2.138-2.651 


3815-4439 


30-55 


Q0453-423 


2.658^^ 


2.359-2.588 


4084-4362 


90-100 






2.091-2.217 


3758-3911 


60-100 


HE1347-2457 


2.609<= 


2.048-2.553 


3705-4319 


85-100 


Q0329-385 


2.434 


1.902-2.377 


3528-4105 


50-55 


HE2217-2818 


2.413 


1.886-2.365 


3509-4091 


65-120 


Q0109-3518 


2.405 


1.905-2.348 


3532-4070 


60-80 


HE1122-1648 


2.404 


1.891-2.358 


3514-4082 


70-170 


J2233-606 


2.250 


1.756-2.197 


3335-3886 


30-50 


PKS0237-23 


2.223*= 


1.765-2.179 


3361-3865 


75-110 


PKS1448-232 


2.219 


1.719-2.175 


3306-3860 


30-90 


Q0122-380 


2.193 


1.700-2.141 


3282-3819 


30-80 


QllOl-264 


2.141 


1.880-2.097 


3503-3765 


80-110 






1.659-1.795 


3233-3398 


45-75 



1-3 
1 
1 

1-2 
1-2 

2 
2-4 
1-2 
1-2 

2 
2-3 
1-2 

2 
1-2 
2-4 
1-2 
2-4 
2-4 
1-2 
2-3 



2288 

3457 

3754 

^3200 

3025 
3157 
3022 

2237 

<3050 

2471 

2163 

< 1629 

< 1750 
<3050 
<3050 
<3052 

2597 



two sub-DLAs at z =2.768 & 2.638 
a sub-DLA at z =3.087 

multiple associated systems 

associated system at 4513.7 A 
a sub-DLA at z =2.305 
only for the H I opacity 



a sub-DLA at z =1.673 



a sub-DLA at z =1.839 
only for the H I opacity 



'^ The redshift is measured from the observed Lya emission line of the QSOs. The redshift bas ed on the emission l ines is known to be 
unde r-estimated compared to the one measured from the absorption lines of the host galaxies IITvtler fc Fanlll992t IVanden Berk et al.l 
[2003). 

*= Rough estimate of the continuum fitting uncertainty. It varies across the spectrum. 

'^ The wavelength of the Lyman limit for each spectrum is defined here as the wavelength below which the observed fiux is zero. This does 
not necessarily correspond to the Lyman limit of an identified Lyman limit system/sub-damped Lya system. Whenever a FOS/HST or 
STIS/HST spectrum is not available, the Lyman limit is assumed to occur at the shortest observed optical wavelength. FOS/STIS spectra 
from the HST archive are available for the following QSOs (any snap-shot survey or spectra with very low S/N are not used): Q0055— 269: 
FOS (PI: Burbidge, Proposal ID: 3199); HE2347-4342: STIS (PI: Heap, Proposal ID: 7575; PI: Kriss, Proposal ID: 8875), FOS (PI: 
Reimers, Proposal ID: 6449), GHRS (PI: Reimers, Proposal ID: 6449); Q0002-422: FOS (PI: Rao, Proposal ID: 6577); Q0453-423: FOS 
(PI: Rao, Proposal ID: 6577); HE1347-2457: STIS (PI: Webb, Proposal ID: 9187); HE2217-2818: STIS (PI: Webb, Proposal ID: 9187); 
Q0109-3518: STIS (PI: Webb, Proposal ID: 9187); HE1122-1648: STIS (PI: Baldwin, Proposal ID: 9885), FOS (PI: Reimers, Proposal 
ID: 5950); J2233-606: STIS (PI: WiUiams, Proposal ID: 8058); QllOl-264: FOS (PI: Bahcall, Proposal ID: 5664) 
'^ In spectra with a sub-damped Lya system, we discard at least 50 A on each side of the centre of the sub-DLA system. 
"= The emission redshift is uncertain due to absorption systems at the peak of the Lya emission line or the occurrence of multiple peaks. 
^ Th e spectrum shows very strong Ovi absorption blended with two saturated Lya absorption systems at 4012—4052 A (jFechner et al.l 
[2OOJ). Since the line parameters for these Lya systems cannot be well constrained (their corresponding Ly/3 is below the partial Lyman 
limit produced by the z ~ 2.738 systems), we discarded this wavelength region. 



20031: ISeliak et al.ll2003l: ISongailal [20o3 : iTvtler et~alll2004 
Kirkman et al.l l2005l : iLidz et al.l |2006|) and we only sum- 



marise a few important points here. 

First it should be noted that there are fundamental 
differences in the way continuum fitting is performed for 
low/intermediate resolution, low S/N data such as the SDSS 
Lya forest data and high resolution, high S/N data obtained 
with high resolution spectrographs such as VLT/UVES 
and Keck/HIRES. For low/intermediate resolution, low S/N 
data the unabsorbed continuum level is normally estimated 
using a simple extrapolation from the much less absorbed 
region of the spectrum redward of the Lya emission line at 
wavelengths Arcst-framo ^ 1250 A. The continuum is thereby 
assumed to be a power law with some superimposed emis- 
sion lines (Press et al. 1993; Bernardi et al. 2003). 

For high resolution, high S/N spectra the contin- 
uum is normally fitted by locally connecting apparently 
absorption- free regions |McDonald et al.l |200G| : iKim et al.l 
I2OO4I : iKirkman etall |2005| ). This detailed local fitting of 



a continuum is time consuming and ceases to work well 
once the redshift of the QSO becomes large (zcm > 4), 
where severe blending makes it impossible to identify any 
un-absorbed regions. In the redshift range considered here, 
1.5 < Zcm < 3.5, however, this method still works well. We 
used it to determine an initial guess for the continuum level 
for the 18 spectra in our sample. Due to the high resolution, 
high S/N of our data, emission lines both weak and strong 
(mainly near 1073 A and 1123 A in rest-frame; Bernardi et 
al. 2003; Tytler et al. 2004) are easily identified in the for- 
est, despite the lack of an appropriate flux calibration. The 
strong, broad ozone absorption bands at ^ 3400 A only af- 
fect the Ly a forest at the lo wer end of the redshift range of 
our sample (|Schachterlll991h . However, this is compensated 
for by the rather low density of absorption features at low 
redshift which leaves more absorption-free regions to fit the 
continuum in our high resolution, high S/N spectra. 

Most of the spectra were not fiux-calibrated, and the 
fiat-fielding procedure was not always ideal, and as a conse- 
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quence there were some spurious broad features in a num- 
ber of the spectra. Rather than attempt to fit the entire 
spectrum using an impractically high order function, the 
spectrum was divided into chunks. The size of these was 
determined interactively, and depended on the redshift and 
the presence of strong absorption systems. Typically these 
chunks were 150-300 A long in the Lya forest part of each 
spectrum, and at longer wavelengths ranged from ~ 200 A 
in QSO emission lines up to ~ 1000 A elsewhere. Legen- 
dre polynomials of order ~ 20 in the Lya forest, and up to 
^ 200 at longer wavelengths, were fitted to each chunk, us- 
ing the IRAF CONTINUUM/ECHELLE procedure. These 
local continua were then combined to produce a single con- 
tinuum for the entire spectrum, and any small discontinu- 
ities at the boundaries were adjusted manually to produce a 
smooth result. We will call the continuum obtained in this 
way the initial contirmum, d. 

As we will describe in the next section we have per- 
formed a full joined Voigt profile analysis of the Hi and 
metal absorption in the spectra in order to obtain absorp- 
tion line parameters (the redshift z, the column density TV 
in cm~'^ and the Doppler parameter or b parameter in km 
s~^). The Voigt profile analysis is sensitive to the assumed 
continuum and the simultaneous fitting of different transi- 
tions caused by the same ion often reveals where the con- 
tinuum should lie in absorbed regions of the spectrum. We 
have therefore adjusted our initial continuum d when we 
fitted absorption features with the Voigt profile fitting rou- 
tine VPFITIj. The fitting procedure and the adjustment of 
the continuum level are very similar to the one described 
in Carswell, Schaye & Kim (2002) and will be discussed in 
more detail in Kim et al. (2007, in preparation). 

For each spectrum, we first searched for metal absorp- 
tion using the entire range of the available spectrum. The 
identified metal lines were the first to be fitted. During this 
process, the continuum was adjusted to obtain acceptable 
ion ratios. If the metal lines were blended with H I absorption 
features, the Hi and metal lines were fitted simultaneously. 
Once all identified metal absorption was fitted we then fitted 
the rest of the absorption features assuming they are due to 
Hi absorption. For this we used higher-order Lyman series 
lines whenever they were available to constrain saturated 
lines. In doing so, we further adjusted the continuum level 
to achieve a satisfactory fit for the available Lyman series. 
Using this second continuum estimated from the first profile 
fitting procedure, the profile fitting was repeated and the 
second continuum was checked manually again. This pro- 
cedure was repeated several times, until we were satisfied 
with the results. The fitting was performed both by R. F. 
Carswell and T.-S. Kim independently, and the final fitting 
was carried out by T.-S. Kim. Note that we did not account 
for the possibility of an extended smoothly varying compo- 
nent of weak absorption often referred to as Gunn-Peterson 
absorption. We further assumed that the response function 
of UVES is smoothly varying for the non-calibrated spec- 
tra as seen in a smoothly varying master response function 
provided by ESO. 

The final adjusted continuum obtained from the fitting 
procedure was then used to obtain the final normalised spec- 



* Carswell et al.: |http://www. ast.cam.ac.uk/~rfc/vpfit. html] 
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tra used in this study. When we use the term "continuum" 
in the following, it always refers to this final continuum, 
Cf. The difference between C/ and d is small and usu- 
ally restricted to some limited wavelength regions, except in 
the regions around Lyman limit systems and sub-damped 
Lya systems where the difference becomes rather large. We 
stress once again th at these final cont inua are different from 
the continua used in lKim et al. (2004) which were estimated 
using the d procedure only. Note also that these final con- 
tinua are not necessarily always completely smooth since in 
some regions of the spectrum continuum adjustments have 
been applied using a straight line to get reasonable column 
density ratios for difi'erent transitions. These small adjust- 
ments are required mainly because of the characteristics of 
the un-calibrated spectra, especially at shorter wavelengths 
at ^ 3400 A. The number of adjustments depends on the 
QSO and varies from none to several. Using a comparison 
between calibrated and non-calibrated spectra, we were able 
to correct some non-smooth features caused by instrumental 
arti facts. 

iKirkman et al.l (|20050 have performed a very detailed 
study of the localised continuum fitting procedure used 
here using 24 high resolution, high S/N Keck/HIRES spec- 
tra. They tested their continuum fitting uncertainties us- 
ing real and artificia l spectra fitted by 4 different people. 
iKirkman et al.l (|2005l ) found noticeable differences between 
the fits by different people. Similarly, R. F. Carswell and 
T.-S. Kim did not agree on some parts of continuum fits of 
the spectra, confirming that such a continuum fitting pro- 
cedure is not fully objective. However, given the complexity 
of the problem we cannot see a better way of doing it. We 
will later quantify the effect of continuum uncertainties on 
the flux probability distribution function and the effective 
optical depth due to Hi absorption in Section |4] and [5] 

The continuum uncertainty depends strongly on the 
S/N of the spectrum as listed in Table [T] The higher the 
S/N is, the smaller the continuum uncertainty is. In the 
upper panel of Fig. [2] we show the calibrated spectrum 
of Q0420-388, one of our best spectra with a S/N =100- 
140. The lower panel shows the non-calibrated spectrum of 
PKS0329— 255, one of the two worst quality spectra in our 
sample. For illustrative purpose we show the effect of a 5% 
decrease (C-s) and increase (Cs) of the continuum level 
by the dashed curves. It is obvious that C5 and C_5 dra- 
matically over/under-estimate the continuum level in the 
case of Q0420-388. Even for PKS0329-255, C5 and C-5 
over/under-estimate the continuum in most parts. Unfortu- 
nately, a rigorous quantitative assessment of the continuum 
uncertainty is not possible given the complex nature of the 
continuum fitting process. 

In Table[T]we give rough estimates of the continuum un- 
certainty obtained from absorption-free regions of the spec- 
tra. For this we compared the highest and the lowest flux 
levels in absorption-free regions to the average flux level in 
the same regions, disregarding localised excursions in only 
one pixel. Note that the continuum uncertainty is not con- 
stant across the spectrum. Note further that due to the pos- 
sible presence of extended featureless absorption, the Gunn- 
Peterson absorption, the continuum used in this study is 
more likely to be an underestimate than an overestimate of 
the true continuum. 

The two lowest quality spectra in the sample are 
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Figure 2. One of the best (Q0420-388 with Zem =3.166, upper panel) and one of the worst (PKS0329-255 with Zem =2.704, lower 
panel) quality spectra in our sample. The solid curves show our final continuum fit, while the dashed curves show a continuum level 
increased/decreased by 5%. The dot-dash curves which are almost indistinguishable from the solid curves are the initial continuum fit, 
d. The metal absorption has not been removed in both spectra. The spectrum of Q0420— 388 is flux-calibrated and has one of the 
highest signal-to-noise ratios. We estimate the uncertainty of its continuum fit to be less than 1%. The spectrum of PKS0329— 255 is 
not flux-calibrated and has low signal-to-noise (S/N ~ 30-40) in most parts of the spectrum. It also shows the largest residuals from 
the fitted line profiles and the largest zero flux offset. We estimate the continuum uncertainty to be 3—4% in the lower signal-to-noise 
regions. The vertical thin-dashed lines indicate the minimum and maximum wavelengths used in the analysis. 



PKS0329-255 (lower panel of Fig. [2]) and J2233-606. 
At 3700 A, J2233-606 has a S/N of 30-35, and for 
PKS0329-255 the S/N is generally ~ 30-40. The contin- 
uum uncertainties are ^ 3-4% for the lower S/N regions 
and ^ 2-3% elsewhere. PKS0329— 255 also has the largest 
zero flux-level offset (see the next subsection) in our sample. 
PKS1448-232 and Q0122-380 also have continuum uncer- 
tainties of ~ 3-4% in regions where the S/N is ~30-35, at 



^ 3550 A and at ^ 3450 A, respectively. Fortunately most 
our spectra have higher S/N, and the continuum uncertain- 
ties are generally around 1-2%. 



2.3 Uncertainty of the zero flux level 

As discussed in iKim et al] ((200I), the UVES standard 
pipeline reduction program returns spectra where the sky 
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Figure 3. Various examples of metal absorption in the Lya forest region of the spectra in our sample. Thin and thick curves represent 
observed spectra and fitted line profiles, respectively, a) Region A in the upper panel shows an isolated metal absorption system, Si IV 
1393 at 2 = 2.221 towards HE0940-1050. Fell 1608 at z = 1.789 at 4485.4-4487.2 A is not shown in order to make the figure simpler. 
The lower panel shows the spectrum after subtracting the metal absorption in region A and adding noise from nearby absorption-free 
regions, b) The thick solid curve in the top panel shows Fen 1608 at 2 = 1.919 (region B) and Fen 1611 at 2 = 1.919 (region C) in the 
spectrum of HE0940— 1050. The line parameters for both absorption systems have been obtained from other less strongly blended Fell 
transitions at other wavelengths. The fitted H l-only profile is shown in the middle and the difference between the observed profile and 
the fitted Hi profile is shown in the bottom panel, c) Regions D, E and F in the top panel mark absorption by Civ 1548 at 2 = 1.732, 
Civ 1548 at 2 = 1.735 and a mixture of Cll 1334 at 2 = 2.175 and Civ 1550 at 2 = 1.732 in the spectrum of HE0940-1050. The 
second panel shows the regions of this part of the spectrum not contaminated by metals. The fitted H l-only profile and the difference 
between the fitted H l-only and the observed profile are shown in the third and fourth panel, respectively, d) The absorption profile of a 
metal absorption system at 2 = 2.045 in the spectrum of Q0109— 3518. The wavelength scale in this and the next two panels has been 
transformed to a corresponding Doppler velocity. The profiles of the Si IV and Sill absorption suggest that two Sim 1206 components 
exist at km s~^and ~ 54 km s~^ (indicated by the arrows) which are blended with saturated Hi absorption, e) The velocity profile for 
Si III 1206 in the same system as in d). The top panel shows the Si III region at 2 = 2.045. The superimposed line is the H I Lyo profile at 
2 = 2.022. The second panel is the Hi Ly/3 profile at 2 = 2.022. The profile fit used both Lyo and Ly/3 simultaneously, assuming no Sim 
(at z = 2.045) contribution. The third panel shows the Sim absorption profile with the two components at and ~ 54 km s~^, assuming 
the same line strength ratio for these two components as estimated from the resolved Si II and Si III absorption. The fourth panel is the 
generated profile of the Hi (2 = 2.022, top panel) and Sim absorption (2 = 2.045). The bottom panel is the difference between the 
observed and the generated Hi + Sim profiles, f) The velocity profiles of a metal line system at 2 = 2.907 towards PKS2126— 158. The 
second panel shows the Sim profile assuming the contribution from the superimposed Hi at 2 = 2.878 which was estimated from the 
corresponding Ly/3 profile (top panel). The third panel shows the Sim profile assuming no contribution from Hi. The weak, broad Sill 
1260 absorption (the strongest Si II transition in the optical spectra) in the fourth panel is assumed to be a non-detection. 
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level is somewhat under-subtracted. This problem becomes 
more severe at shorter wavelengths (^ 3400 A). The under- 
subtraction of the sky level results in a non-zero flux level 
within the troughs of saturated lines. The offset from zero 
flux depends on signal-to-noise and wavelength. For increas- 
ing signal-to-noise and longer wavelengths the offset from 
zero becomes smaller. The worst case is PKS0329— 255 with 
a typical offset, Ai^zcro = 0.015 in a spectrum normalised 
to unit continuum. Note that this QSO also has one of the 
worst overall continuum uncertainty. Most spectra have an 
offset of much less than Ai^zcro = 0.01. An offset from zero 
flux can lead to significant problems for the Voigt profile fit- 
ting. Since the flux in saturated regions does not go to zero, 
VPFIT normally adds many narrow components whose flux 
minima do not reach F ^ Q instead of fitting the saturated 
profiles with fewer strong components, unless one specifi- 
cally allows the zero level to vary. Fortunately, this problem 
has a negligible effect on the statistics of the flux distribu- 
tion presented here, as will be discussed in more detail in 
Section l44l 



3 A FULL VOIGT PROFILE ANALYSIS OF 
THE H I AND METAL ABSORPTION 

Most simulations and models of the Lya forest account only 
for the distribution of Hi. In reality, the Lya forest con- 
tains significant absorption from intervening metals in the 
IGM. These metal absorption lines are obviously interest- 
ing in their own right and provide a wealth of information 
on the spectrum of the metagalactic UV background and 
the metal enrichment of the IGM by galaxies. For a statis- 
tical analysis of the Hi absorption aimed at studying the 
underlying matter distribution, they are, however, a source 
of contamination which leads to additional uncertainties. In 
this case it is therefore important to identify the metal lines 
in the forest and to remove them from further study, or at 
least to quantify their effect on any statistic investigated. 
Most studies so far have dealt with this problem by exclud- 
ing spectral regions with strong metal contamination from 
the analysis, e.g. a stra tegy adopted bvlRauch et al.l l|l997h . 
iMcDonald et al.l l|2000l ) and lLidz et al.l (|2006l ). 

The metal contamination is, however, rather widespread 
and varies strongly between different spectra. Excluding 
contaminated regions is therefore problematic. If metal 
lines are isolated, it is straightforward to excise the metal- 
contaminated regions. More often than not, metal absorp- 
tion is, however, blended with Hi absorption at 2 < 2 < 
S.qj. It is then difficult to decide which regions to exclude. 
Since there are only a few QSOs in each redshift bin in this 



^ The amount of the metal absorption blended with H I absorp- 
tion depends on redshift. At low redshift (z < 2.5) the Hi ab- 
sorption is weaker and most metal absorption occurs in region 
with weak or no Hi absorption. At higher redshift (2 > 2.5), the 
number density of H I absorption features increases and the metal 
absorption starts to blend in regularly with the H I absorption. At 
very high redshift (2 > 3.5—4), the density of Hi absorption fea- 
tures becomes very high and many lines become saturated. In this 
case, many metal lines are blended with strongly saturated H I ab- 
sorption and their contribution to the total absorption diminishes 
or becomes negligible. 



and other similar studies, attempts to exclude metal contam- 
inated region are likely to lead to a selection bias, especially 
at 2 ~ 3. 

We take a different approach here, capitalising on a full 
Voigt-profile analysis of the Hi and metal absorption per- 
formed by R. F. Carswell and T.-S. Kim which will be de- 
scribed in more detail elsewhere (Kim et al. 2007, in prepa- 
ration). In this Voigt profile analysis we have identified all 
metal lines in the 18 spectra to the best of our knowledge. 
This allows us to remove their absorption contribution from 
the observed flux distribution. This method produces an es- 
timate of the continuous flux distribution without any ab- 
sorption due to identified metal lines. 

For the identification of metal lines, we made use of 
known properties of metal absorption: 

(i) Metal absorption features tend to be narrower than 
Hi lines. 

(ii) Metal absorption features are usually associated with 
strong, often saturated Hi absorption. 

(iii) In the redshift range considered here, it is unlikely for 
Hi absorption to show corresponding Sill without showing 
Civ, Cii, Siiv or Sim. In short. Hi absorption with no 
corresponding C iv is unlikely to show other metal lines, 
especially for the weaker absorption systems with column 
density iVni ^ 10"cm"^. 

We started by identifying metal absorption features red- 
ward of the LyQ emission where all absorption lines (apart 
from the telluric lines, which were omitted) are due to met- 
als. Most of these are Civ, Siiv, Sin, Alii, Aim, Mgii and 
Fe II. The rest-frame wavelengths and os cillator s t rengt hs for 
transitions from these ions are given bv lMortonI l|2003r ). Us- 
ing these lines as a guide, we looked for other metal lines 
at the same redshift, such as C ill, O vi, C I. In a second 
step we searched for metal lines associated with H I absorp- 
tion with any column densities in the UVES spectra, i.e. at 
> 3050 A. Thirdly we looked for possible doublet candidates 
in the forest. In a fourth and final step we looked for possi- 
ble metal lines associated with saturated H I absorption or 
with the Lyman discontinuity shown in the HST data listed 
in Table [TJ i.e. at ^ 3050 A. 

With information on absorption systems at 2 < 1.5 
available for only half of the QSOs (see Table [l| and none 
of them observed at as high a resolution and S/N as the 
UVES data, it is very difficult to identify all the metal lines 
in the forest. Severe line blending often makes the identifica- 
tion of metal lines difficult. Sometimes an absorption feature 
first thought to be due to H I did not show a corresponding 
Ly/3 line of the expected strength when we attempted to 
fit higher-order Lyman series lines. Any excess absorption 
at the Lya wavelength was then considered as being due to 
yet to be identified metal lines. 

All other absorption not identified as being due to met- 
als we have assumed to be due to Hi. There were a few 
complexes of strong clustered rather narrow lines (Doppler 
parameters less than 15 km s~^) which we suspect to be 
unidentified metal lines. We have flagged these as potential 
metal lines. These occasions, however, were rare. Note that 
some of the weak narrow lines which we considered as due to 
H I absorption could still be either weak, unidentified metal 
lines or noise peaks, while stronger metal lines are almost 
complete in their identification. 
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The different metal lines and H I were all fitted inde- 
pendently. Only the transitions produced by the same ion 
of the same redshift systems were required to have the same 
redshift and the Doppler parameters. Sometimes part of a 
given metal line is blended with strong H I absorption which 
makes it difficult to separate the metal and Hi lines. Only 
in such circumstances, the redshifts and the Doppler pa- 
rameters were tied with the ones measured from the other 
clean metal lines within the same ionisation group (such as 
Mgll, Fen, Cll and Sin for the low- ionisation group), sim- 
ilar to the method usually employed in the metal analysis 
of damped Lya systems. Fortunately, most common metal 
lines embedded in the forest, such as Mgll, Fell, Sill, Civ, 
are multiplets. Identifying these lines is more reliable than 
single-transition lines, such as Sim 1206. 

While we are likely to have identified almost all strong 
metal lines, we would like to stress again that we will in- 
evitably have ascribed some weak lines wrongly as being 
due to Hi. 

Fig.|3]shows some examples of the metal lines embedded 
in the forest. It also illustrates how we have subtracted the 
metal contributions from the fiux distribution. Thin lines 
are the observed profiles (Hl-|- metal lines), while the thick 
lines are the fitted Hi or metal lines. The upper panel of 
Fig. [3] a) shows a single isolated metal system and the lower 
panel shows the same region after removing the metal lines 
and adding the noise estimated from nearby, absorption-free 
regions. About 40—50% of metals are isolated at z ~ 2, the 
fraction decreases to 20—30% at z ~ 3, due to increased 
Lyman line blending. 

The top panel of Fig. |3] b) shows a moderate-strength 
metal line blended with Hi absorption. Such embedded 
metal lines have been identified by other transitions falling 
outside the forest region or in regions of the forest where the 
H I absorption is weak. The middle panel shows the H l-only 
absorption profile generated from the parameters obtained 
from the line fitting. The lower panel shows the difference 
between the flux in the top (Hl-|-metal) and middle panels. 
The difference is of the same order as the pixel noise. If metal 
absorption of weak to moderate strength is blended with sat- 
urated H I absorption, the metal absorption contributes very 
little to the combined absorption proflle. 

Fig. [3] c) shows a very common configuration of metal 
absorption. Approximately 45% of metal absorption systems 
a.t z ^ 2 and ^ 75% at 2 '^ 3 look similar to this. The top 
panel shows a region of a spectrum overlaid with the metal 
absorption profile of identified metal lines. In region D the 
metal absorption is blended with moderate-strength H I ab- 
sorption. If we wanted to excise the metal absorption it is 
not obvious if one should cut out the entire H I absorption 
complex from 4226 A to 4236 A or only the region where the 
metals affect the Hi profile from 4229.5 A to 4231.2 A as 
shown in the second panel. The third panel shows instead the 
Hl-only absorption profile generated from the parameters 
obtained from our line fitting procedure. The bottom panel 
shows the difference between the H l and the H l-|- metal pro- 
files. 

The most difficult metal absorption features to obtain 
a reliable line fit for in the Lya forest region of the spec- 
trum is the absorption due to the single transition of Sim 
at 1206.5 A (see McDonald et al. 2006). Sometimes we could 
successfully fit Si ill absorption profiles blended with H I ab- 



sorption using other metal lines, such as Lya, Ly/?, Sill, 
Si IV, Aim, Alii, Cii, Civ, and Mgii. Sometimes, however, 
we could fit only part of a Sim profile. In such cases some 
contamination of the final Hi profile is unavoidable, since 
we could not subtract all of the Si iii contribution from the 
H l-|-Si III absorption feature. An example is shown in Fig. [3] 
d) and e). 

Fig- Eld) shows a typical metal line system at z = 2.045 
towards Q0109— 3518 where Sim is blended with Hi ab- 
sorption at z = 2.022. Note that we plot relative velocities 
here instead of wavelength as is customary for strong metal 
absorption complexes. The low (high) ionisation lines Sill 
(Siiv) suggest that there should be two components of Sim 
at km s~^and ~ 54 km s~'^in the saturated Hi profile 
at z = 2.022, as indicated by the arrows. The top panel of 
Fig.[3]e) shows the Sim region of the system shown in panel 
d). Superimposed is the Hi Lya profile at z — 2.022, which 
was fitted together with the corresponding Ly/? absorption 
(second panel), assuming no Sim (at z = 2.045) contribu- 
tion. The third panel shows the Sim profile with the two 
components at 1; = and ~ 54 km s~^ (indicated by the ar- 
rows). These two components were generated assuming the 
same relative line strengths between all successfully fitted 
Si II and Si ill components. In the 4th panel the H I Lya (at 
z — 2.022) + Sim components (at z = 2.045) are superim- 
posed. Note that the fit parameters of the Hi profile are the 
same as in the first panel. The bottom panel shows the dif- 
ference between the observed and the fitted Hi Lya-I- Sim 
profiles. The figure demonstrates that if the H I absorption 
is saturated, then even strong metal lines do not contribute 
significantly to the overall profile, unless the metal absorp- 
tion extends to the wings of the saturated Hi absorption 
profiles. 

Fig. [3] f ) shows a velocity plot of a metal line system 
at 2 = 2.907 towards PKS2126-158. It illustrates the un- 
certainty of the profile fit for a single transition Sim 1206 
and a very weak line. The second and the third panels show 
the Sim 1206 absorption, assuming a Hi contribution at 
z = 2.878 constrained by the corresponding Ly/3 (the top 
panel indicated by a thick tick mark) and assuming no H I 
contribution, respectively. In reality, its true column den- 
sity could range between the ones obtained with these two 
assumptions. Similarly, the weak, broad feature at the ex- 
pected location for Sill 1260 (the strongest Sill transition 
available in the spectra) in the fourth panel could be a real 
detection blended with weak Hi (the upper limit for A'^sin 
can be estimated from non-detection of other Si 11 transitions 
above the Lya emission) or a weak broad Hi. Since we do 
not want to over-identify metals and want to be conservative, 
we adopted the lowest Si ill 1206 column density (assuming 
a contribution by Hi) and considered Sill 1260 to be not 
detected in this case. Only when it was not possible to con- 
strain H I from higher order Lyman series, we assumed that 
the absorption was due to metals only. Non-detection in Ly/3 
in our spectra typically corresponds to A^hi ^ lO'^^'* cm~^ 
assuming 6 = 20 km s~^ (or F ^ 0.8 in Lya). Fortunately 
there are not many such cases in our sample. In Table IA2I 
in Appendix A we list uncertain line fits. 

In summary, for each QSO we have obtained a Lya 
forest spectrum free of identified metal lines as follows: 

• for isolated metal lines the spectrum was replaced by 
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continuum with a noise level estimated from nearby contin- 
uum regions; 

• where metal lines and Hi lines are blended, that part 
of the spectrum was replaced with the model H I lines de- 
termined from fitting Voigt profiles to the heavy element 
and Lyman lines simultaneously, with noise using estimates 
appropriate to the final flux levels determined from nearby 
regions. 

In Fig. |4] we show the continuum fitted spectra of our 
complete sample together with the metal contribution to 
the total absorption to give a general impression of the metal 
contamination. Note that the metal contribution to the total 
absorption differs significantly from the metal-only absorp- 
tion as metal absorption blended with strong Hi has little 
effect on the total absorption profile. 



4 STATISTICAL ERRORS AND SYSTEMATIC 
UNCERTAINTIES OF THE FLUX 
PROBABILITY DISTRIBUTION 

4.1 Statistical errors 

One of the main aims of the paper is to estimate the Hl- 
only absorption flux PDF from the normalised spectra of our 
sample. The PDF of the flux is simply the number of pixels 
which have a fiux between F and F -\- A_F for a given flux 
F divided by the total number of pixels (IJenkins fc Ostriker 
1991"; Rauch ct al. 'l997|; iBrvan et al]|l999l : lMcDonald et al 
2000 ; Kim ct al. 2001 ). 



Before presenting our new measurement of the fiux PDF 
from our full sample we will discuss the statistical errors of 
the measured fiux PDF and investigate some of the system- 
atic uncertainties_discussed in Section [2] on the fiux. Follow- 
ing |McDonaidilaL| (|2000i ). we calculate the fiux PDF for 
bins of width AF = 0.05. Pixel with flux level smaller than 
F = 0.025 or greater than F — 0.975 have been allocated to 
the F = bin and the F = 1.0 bin, respectively. 

To estimate the errors of the PDF both for a single QSO 
and for a full sa mple, we us e d a m odifled jackknife method 
as presented by iLidz et all (|2006l '). We briefly outline the 
method here for clarity. We first calculate the fiux PDF of 
the full sample. This sample was then divided into n^ chunks 
with a length of ~50 A. If the PDF estimated from the full 
sample at the fiux bin Fi is P{Fi) and the PDF estimated 
without the k-th chunk at the flux bin Fi is Pk{Fi), then the 
covariance matrix cov(i, j) between the PDF in a flux bin 
Fi and the PDF in a flux bin R was calculated as 



cov(i,j)=^[P(FO 



Pk{F)][P{F,) ^ P,{F,)], 



(1) 



and the variance at a given fiux level is given by the diagonal 
terms of the covariance matrix af = cov(i, i) for a fiux bin 
Fi. We checked that this modifled jackknife method is not 
sensitive to the length/number of chunks for a given sample 
size. As expected, the errors are larger when the number of 
pixels in the sample is smaller: ~15% for Q0420— 388 with 
13561 pixels vs ~7% for the fuU < 2 > = 2.94 sample with 
34265 pixels at F = 0.5. We compared the errors obtained 
with the modified jackknife method with the errors obtained 
by 500 bootstrap realisations of chunks of 100 pixels (or 5 A) 



used by iMcDonald et all l|2000l ) and ISchave et all (|2003h 
(see Section [5. 2 1 for more details for the bootstrap method). 
Both methods give comparable error estimates, while the 
Poisson errors (i.e. those based on the square root of the 
number of pixels) tend to be ~ 35% smaller. 

4.2 Continuum fitting 

We now move to a discussion of the effect of systematic 
uncertainties on the flux PDF. We start with the effect of 
continuum fltting uncertainties. In Fig. [S] we show the fiux 
probability distribution of the spectrum of Q0420— 388 in- 
cluding the metal absorption (see also Fig. |6]) with our final 
continuum fit C/, our initial continuum fit d and four fur- 
ther continua where we have applied a wavelength indepen- 
dent offset of the continuum level of ±1% (Ci and C_i) and 
±5% (C5 and C-s). The dotted, dashed, solid, dot-dot-dot- 
dashed and dot-dashed curves show the PDF with C5, C_5, 
Cf, C'l and C_i, respectively. The fiux PDF of the spectra 
with the initial continuum fit d is almost indistinguishable 
from the PDF of the spectra with the final continuum fit 
Cf. As expected changing the continuum level affects the 
PDF most strongly at a flux level oi F ~ 1, shifting the 
corresponding peak in the PDF. Continuum fitting uncer- 
tainties also have a moderate effect on the slope of the PDF 
at fiux levels 0.65 < F < 1.0. There is little effect at lower 
flux levels as regions of saturated or very strong absorption 
are not affected by an o);er-/under-estimated continuum as 
much as regions of weak absorption. 

Note again that a systematic change of the continuum 
level by 5% is a gross overestimate of the actual continuum 
fitting uncertainty for most of our spectra. We choose this 
value here simply to demonstrate the effect more clearly. 
Most spectra in our sample have a continuum uncertainty 
of ~ 1-2%, and the changes to the PDF are small. 



4.3 Metal contamination 

As apparent from Fig. |4] the metal contribution to the ab- 
sorption varies significantly between different spectra. In Ta- 
ble IA4I in Appendix A we quantify the metal contamination 
of the H H-metals effective optical depth, i.e. observed val- 
ues before removal of the metal absorption, estimated using 
the effective optical depth after removal of the metal absorp- 
tion. The values vary from 0.5% to 28% (note that in the 
case of Q0055— 269, the metal removal is incomplete due to 
the limited wavelength coverage from 3200—6810 A). There 
appears to be no obvious trend of the extent of the metal 
contamination with other parameters, such as z, except that 
spectra containing Lyman limit systems or sub-damped Lya 
systems tend to show a larger absorption contribution by 
metals. The mean metal contamination is ~12%. This is con- 
sistent within the errors quoted for statistical estimates of 
the m etal contamination: ~ 19% at z = 1.9 by Tvtler et all 
(J200i) and ~ 10% at z = 2.7 bv lKirkman et all l|2005l F[ 



Fig. [S] illustrates the effect of absorption by metals on 

^' Both removed the metals in the forest statistically. Using pub- 
lished line parameters redwards of the Lya emission of QSOs at 
1.7 < Zom < 3.54, they estimated the amount of metals as a 
function of rest-frame and observed wavelength. 
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Figure 4. Normalised observed spectra and the additional metal contribution to the absorption for our sample of 18 VLT/UVES spectra 
(against rest-frame wavelength). The regions with no flux, such as at ~ 1315 A of PKS2126— 158, are due to the wavelength gaps caused 
by using three separate CCDs in the dichroic setting. The normalised observed spectra are shown by the thick solid curves in the upper 
panels for each QSO. Note that the lower panel is not the fitted metal absorption. It is the difference between the observed flux and the 
fitted H I absorption and thus shows the additional absorption by metals which is significantly smaller than the absorption by metals 
would be in the absence of H I absorption. The four vertical dotted lines indicate the rest-frame wavelength of Ly/3, Lya plus 1268 A 
and 1380 A. Note that the weak metal absorption at 1268-1380 A does not necessarily indicate a low metal contamination in the forest 
(see e.g. the spectra of HE0940— 1050 and HE1347— 2457). The most common metal species found in high-z QSO spectra is Civ. The 
high column density absorption systems showing strong Civ (1548 A and 1550 A) and Si IV (1393 A and 1402 A) absorption features 
usually show other metal lines. These high column density absorption systems occur randomly along each sightline, i.e. at different z for 
different sightlines. Therefore, stronger transitions e.g. Fell 2382 & 2600, Sill 1190, 1193 & 1260, or Mgll 2796 & 2803, which are usually 
associated with high column density systems, could be present in the Lya forest region without there being any corresponding strong 
metal lines in the 1268-1380 A range. The metal contribution in the spectrum of Q0055— 269 forest could be significantly underestimated 
since the wavelength coverage in the red is rather limited, only up to 6809 A (observed). 
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Figure 5. The upper panel shows the flux PDF of Q0420-388 
(2.480 < z < 3.038) for 6 different continuum levels. The solid 
curve is the PDF for our best fit final continuum Cf, while the 
curve almost indistinguishable from the solid curve is the PDF 
for the initial continuum d. The dotted/dashed curve are the 
PDF for a continuum level increased/decreased by 5% (C5 and 
C-s), while the dot-dot-dot-dashed/dot-dashed curve are for a 
continuum level increased/decreased by 1% (Ci and C_i) The 
errors were estimated using a modified jackknife method by Lidz 
et al. (2006) as described in the text. Different continuum fits also 
shift the peak of the PDF, as indicated by the arrow for C-5. The 
lower panel shows the following ratios PDF(j_ /PDFc (dashed 
curve) and PDFcj^/PDFc- (dotted curve). The errors in the 
lower panel are similar for both curves but plotted only once 
for clarity. The errors correspond to the combined errors of both 
PDFc^5 and PDFcj. ■ 



the flux PDF. The upper panel shows the PDF of two arti- 
ficial spectra which were generated from the fitted line pa- 
rameters of PKS2126-158. The solid curve is the PDF after 
removal of the identified metal lines while the dotted curve 
is the PDF before removal of the metal absorption. The 
bottom panel shows the ratio of the two. Unlike continuum 
uncertainties metal absorption affects the flux PDF mostly 
at flux levels in the range < F < 0.6 even though there 
is also a small reduction of the flux PDF at F ~ 1. Note 
that the latter effect is likely to be underestimated Eis weak 
metal absorption is difflcult to identify. Since the number 
of absorption features which we classifled as suspected metal 
lines are small and usually weak, these lines have a negligible 
effect on the PDF at 0.1 < F < 0.6. 



4.4 Pixel noise and the uncertainty of the zero 
flux level 

Pixel noise will cause a slight smoothing of the flux PDF. 
For high-S/N spectra as in our sample the effect is small 
but noticeable. This is demonstrated in Fig. [T] The upper 
panel shows the flux PDF of PKS0329-255 for the observed 
spectrum with metals (dot-dashed curve) and without met- 
als (solid curve with error bars) . The observed spectrum has 
S/N = 30-55. We generated 4 artificial spectra with differ- 
ent S/N using the fitted line parameters, assuming Gaussian 
noise: the dashed curve is for S/N = 25, the dotted curve 
is for S/N = 50, and the two almost indistinguishable thin 




0.4 0.6 

Flux 

Figure 6. The upper panel shows the flux PDF of PKS2126-158 
with metal absorption (dashed curve) and without metal absorp- 
tion (solid curve). The two spectra were generated using fltted 
line parameters. The lower panel shows the ratio of the flux PDF 
with and without metals. The errors in the lower panel are the 
combined errors. The metal lines contribute to the fiux PDF sig- 
nificantly at flux levels F < 0.6. 



solid curves are for S/N = 100 and S/N = cxQ. Note that 
the generated spectra do not have a offset from the zero 
flux level, i.e. the saturated lines go down to F = 0. The 
bottom panel shows the ratio of the PDF for the artiflcial 
spectrum with S/N = co (solid curve) and the PDF for the 
spectrum with metals (dot-dashed curve) to the PDF of the 
metal-removed spectrum. 

The effect of the signal-to-noise on the flux PDF is 
most evident at flux levels of < F < 0.1 as well as at 
F > 0.9, where it can exceed that from the metal contami- 
nation. At intermediate flux levels, 0.1 < F < 0.9, the effect 
of the signal-to-noise is comparable or smaller than that 
due to metal absorption. It should, however, be noted here 
that the contribution of metal absorption in the spectrum of 
PKS0329-255 is rather smaU (2.8%) and that the spectrum 
has the lowest S/N of our sample. 

In Fig.|5]we show the difference between the flux PDF of 
artiflcial S/N = 00 spectra (i.e. no zero-level offset) and the 
flux PDF of the observed spectra without metal absorption 
for our full sample at three different redshift bins. At flux 
levels of 0.1 < F < 0.8 the difference is less than 1% and 
much smaller than the statistical errors, but outside this 
range the errors increase to a few percent at low flux levels 
and more than 10% at high flux levels. 



^ Note that the noise in the observed spectra is not exactly Gaus- 
sian. The noise at F ~ 0, i.e. the bottom of saturated lines, and 
at F ~ 1 are different, and of course the S/N at F ~ is much 
worse than at F ~ 1. However, for the purpose of illustrating the 
S/N effect on the PDF, the assumption of Gaussian noise is a 
good approximation. 
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Figure 7. The upper panel shows the flux PDFs of the observed 
and artificial spectra of PKS0329-255 for different noise levels. 
The dot-dashed curve is for the observed spectrum including the 
metals, the solid curve with errors is for the observed spectrum 
after the metals have been removed. The four other curves are the 
flux PDFs for artificial spectra generated from the line parameters 
with different levels of Gaussian noise added. The dashed curve is 
for S/N = 25, the dotted curve is for S/N = 50 and the two almost 
indistinguishable thin solid curves are for S/N = 100 and S/N 
= OQ. The S/N = oo and S/N = 100 PDFs show a rather largo 
deviation at F < 0.1 and almost a factor of 2 at i'' = 0.05. This is 
due to the offset of the zero flux level of the observed spectrum (a 
typical offset from zero flux is ~ 0.015) which the artificial spectra 
do not have and the effect of which is not smoothed out in the 
artificial spectra with the highest S/N. The lower panel compares 
the ratio of the fiux PDFs of the S/N = oo spectrum to the metal- 
removed spectrum (solid curve) and the PDF ratio of the metal- 
included spectrum to the metal-removed spectrum (dash-dotted 
curve). The effect of low S/N on the flux PDF becomes noticeable 
at F < 0.1 and F > 0.9. At 0.1 < F < 0.9, the effect of the S/N 
is comparable or smaller than that of the metal contamination. 



5 THE FLUX DISTRIBUTION OF THE FULL 
SAMPLE 

5.1 The flux probability distribution 

Fig. |9] shows the main result of this paper, the flux PDF of 
the observed spectra in our full sample divided into three 
redshift bins (Table lAll in Appendix A) after removal of all 
identified metal lines. The thin, grey curves show the flux 
PDFs of the individual spectra. As found previously there is 
considerable scatter between different lines-of-sight. This is 
mainly due to the occurrence of stron g absorption syst ems 
which are rare in individual spectra ( Viel et al.ll2004al ). A 
considerable path length is therefore required to reach rea- 
sonable convergence to an average flux PDF. This is best 
illustrated by the spectrum of Q0002— 422 which shows the 
most deviant individual PDF compared with the mean PDF 
in its redshift bin. This is probably due to several factors. 
The spectrum falls at the upper end of its redshift bin. It 
has also the shortest usable path length (191 A) and the 
largest number of strong systems per unit redshift in the 
wavelength region used. 

In Fig. [TD] we show the effect of the removal of the 
identified metal absorption. The dashed and solid curves 
in the upper panel show the flux PDF of the spectra with 
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Figure 8. The difference between the fiux PDFs for S/N = oo 
artificial spectra and metal-removed observed spectra for the full 
sample divided into three redshift bins. The errors are those of 
the PDF of the metal-removed observed spectra. The combined 
errors of both PDFs are larger. The effect of lower S/N and the 
offset of the zero flux level on the PDF are negligible compared 
to the statistical errors at flux levels 0.1 < F < 0.8. 



and without the identified metal absorption, respectively. 
The lower panel shows the ratio of the two. Removing the 
metal absorption mainly affects the flux PDF at flux lev- 
els 0.1 < F < 0.8. Without metal absorption the fraction of 
pixels in this flux range is 10—20% lower. The effect of metal 
absorption, which appears to evolve little with redshift, is 
more signiflcant at lower redshift where the Hi absorption 
is smaller. 

In Fig.[Tl]we compare our new measu rement of the flux 
PDF with that of iMcDonald et al.l (|2000l 'l at < z>= 2.41 
and < z> — 3.0. The solid curves are the PDF when met- 
als are included. Unfortunately a comparison at higher red- 
shifts is not possible due to the lack of high-redshift spec- 
tra in our sample. At < z > = 2.41 our measurement is 
about 10-20% lower at flux levels 0.1 < F < 0.5 while at 
<z> = 3.0 our measurement is about 10-30% lower at flux 
levels 0.2 < F < 0.8. We can only speculate here where this 
discrepancy comes from. Part of the difference is probably 
due to the rather crude removal of metal absorption by Mc- 
Donald et al., which is likely to have led to more residual 
absorption by unidentifled metal lines in their spectra. The 
difference appears, however, to be larger than expected due 
to this effect and increases rather than decreases with in- 
creasing redshift. Note that the McDonald et al. sample is 
signiflcantly smaller with a total of 8 spectra and some of 
the discrepancy can probably be explained as being due to 
variations between different lines-of sight (see the grey thin 
curves in Fig. |9} . Differences in the placement of the contin- 
uum level may also play a role. We list the mean flux PDF 
for the flve different redshift bins discussed in this section in 
Table E3l in Appendix A. 

The flux PDF is a statistical quantity which - at least 
in principle - can be easily compared with that of simulated 
spectra. Due to the expected tight correlation between H I 
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Figure 9. The flux PDF (filled circles with errors) of the full sample divided into three redshift bins after removal of the metal absorption. 
The thin grey curves show the PDFs of individual spectra. The la error bars are estimated using a modified jackknife method (Lidz et 
al. 2006). The dashed curve in the <z>= 2.07 bin is the PDF of the spectrum of Q0002-422, which has the highest redshift in this bin 
and the largest effective optical depth. 
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Figure 10. The flux PDF of the full sample divided into three redshift bins before (dashed curves) and after (solid curves) removal 
of the metal absorption. The lower panel shows the ratio of the PDFs before and after removal of the metal absorption. The effective 
optical depth duo to H I absorption is lower at lower redshift and the relative contribution of the metal absorption is therefore larger. 
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Figure 11. Comparison of the flux PDF of our sample divided into two redshift bins with that of McDonald et al. (2000). The solid 
curve and filled circles show the flux PDF of our sample before and after removal of the metal absorption, respectively. The lower panel 
shows the ratio of the flux PDF of McDonald et al. to that of our sample after removal of the metal absorption. 



optical depth, local gas density and temperature, the ob- 
served flux PDF can constrain astrophysic al parameters of 



the I G M as well as cosmoloRic a l parameters llWeinberg et al 



1998; McDonald et al.l 200p|: i Meiksin. Bryan fc Machacek 



200ll : iDesiacgues fc Nus ser 200E|) . The effect of the removal 



of metal absorption and the difference to the published flux 
PDF by McDonald et al. is comparable or larger than the 
effect of changing some of the model parameters within plau- 
sible ranges (e.g. Meiksin et al. 2001; Desjacques & Nusser 
2005). Our new improved measurement of the flux PDF 
should therefore be relevant for attempts to use the flux 
PDF to constrain astrophysical and cosmological parame- 
ters. 



5.2 The Hi effective optical depth 

So far we have concentrated on the flux PDF of our sam- 
ple. The Hi effective optical depth (t^^) is another intensely 
studied quantity which is important for the comparison with 
models of the H I distribution. The H I effective optical depth 

_ oft _ 

is related to the mean flux as exp « i = < exp '^ ' > , where 
< > indicates the mean value averaged over wavelength. 
Note that the effective optical depth is not the average of 
the optical depth. The effective optical depth is the quan- 
tity which directly goes into measurem ents of the ampli- 
tude of the metaga lactic UV background (JRauch et al.ll 19971 : 
iBolton et al.l |2005| ) and plays a crucial role in calibrating 



measurements of the matter power spectrum fr om Lya for- 
est data based on the flux power spectrum (ICroft et al. I 
2002l: 'Scliak ct al.l l2003l : iTvtler et al.ll2004 IViel et al.ll2004bl : 
Lidz et al...2003 r 

As we have taken special care with the removal of the 
metal absorption in our spectra it is worthwhile to revisit 
the effect the removal of the metal absorption has on the H I 
effective optical depth. 

In Fig.[T2]we show the effective optical depth of our ob- 
served spectra before (open squares) and after (flUed circles) 
removal of the metal absorption. The solid lines in both pan- 
els is the power-law fit of r^^ to the optical depth of the high- 
resolution data from our sample and that compiled from the 
literature, rf, = (0.0023 ±0.0007)(1 -hz)^'^'^^^''-^^ (the same 
power- law fit as in Fig. llSp . The err ors of r^'^were est i mated 
with the s ame pro cedure adopted in lMcDonald et al.l (|2000l ) 
and Schave et al. I (l2003iiJ. Each spectrum was divided into 
chunks of 100 pixels (or 5 A). We then performed 500 boot- 
strap realisations, treating each chunk as a one data point. 
The removal of the metal absorption leads to a typical reduc- 
tion in the observed effective optical depth (i.e. Hl-|-metals) 
by 0.5% to 28% with the mean of ~ 12%, albeit as expected 



* Note that the modified jackknifc method using a ~ 50 A-long 
chunk generally gives error estimates of only ~l-2% for each 
QSO. These estimates are too low, since a ±1% continuum un- 
certainty generally gives an error of ~ 2-10%. 
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Figure 12. Left panel; Evolution of the effective optical depth of our full sample after removal of the metal absorption. The horizontal 
and vertical error bars show the redshift range used and the la errors. The la errors were estimated as in McDonald et al. (2000) and 
Schaye ct al. (2003) using 500 bootstrap realisations of chunks of 100 pixels (5 A). Right panel: Comparison of the evolution of the 
effective optical depth of our full sample before (open squares) and after (filled circles) removal of the metal absorption. The solid lines 
in both panels represent a power-law fit to the effective optical depth t|^ j of our sample after removal of the metal absorption and other 
measurements from high-resolution data compiled from the literature at 1.7 < 2 < 4 as in Fig. 1131 t^^ = (0.0023±0.0007)(l + z)^'^^^'^'^^ . 



with a large scatter (see Table lA4l in Appendix A). As men- 
tioned in Section [4.31 these numbers are estimated from the 
effective optical depths of the observed spectra before and af- 
ter removal of the metal absorption. The H I effective optical 
depth is obviously also subject to the same systematic un- 
certainties as the flux probability distribution. The change 
of continuum from C/ to C5 (Ci) increases r^i by 0.049 
(0.010), which is identical for each QSO. The noise and the 
uncertainty in the zero-level flux have a negligible affect on 

The upper panel of Fig. 1131 compares the thi measure- 
ments of our 18 QSOs with those of Schaye et al. (2003). 
Both set of values are measured from spectra where metal 
absorption has been removed. In the case of Schaye et al. 
(2003) this has been done by excising spectral regions con- 
taminated with strong metal absorption. The values are in 
good agreement. Considering that half of the sample of spec - 
tra studied here is part of the sample of lSchave et al. I (120031 ) 
this is not too surprising. For spectra common to both stud- 
ies the differences are, in most cases, less than 10% when 
our wavelength ranges were adjusted to the ones used in 
the Schaye et al. samplqj. Table IX4l in Appendix A lists the 
measurement of Th, as well as the mean flux, its variance 
and the contribution of metal absorption in the forest. The 



^ There are several noticeable differences between the two r^ ^ 
measurements. Since the two r^^values of the metal-included 
spectra are very similar, the difference is mainly caused by 
the incomplete metal removal in the Schaye et al. sample, i.e. 
their t^^ is somewhat larger. QSOs with more than 10% dif- 
ference in rgf^are: J2233-606 at 1.732 < z < 1.963 by ~12%, 
Q0109-3518 at 1.874 < z < 2.120 by ~11% and Q0002-422 at 
2.434 < 2 < 2.710 by ~11%. In Fig. [13] the most noticeable is 
J2233— 606, in part due to the fact that our redshift range used 
in this study differs from the one used by Schaye et al. 



errors are estimated from the 500 bootstrap realisations of 
a chunk of 5 A. 

In the lower panel of Fig. [13] we have compiled a range 
of th, measurements from the literature at 1.7 < ^ < 4 
and compare this with a measurement of Th i from our sam- 
ple after removal of the metal absorption binned in red- 
shift bins with width Az = 0.2 (filled circles with errors). 
Our Tg, is estimated from the mean flux of all pixels in 
each bin, instead of averaging the effective opacities of each 
QSO (see Table [XS] in Appendix A). The errors are esti- 
mated by the 500 bootstrap realisations of 5 A using all pix- 
els in each bin. The open squares, fl lled squares and fllle d 
triangles show the m easur ements of Schaye et al. I (|2003l ) , 
iKirkman et~ai1 (|2005l ) and iMcDonald et all l|200o| y! respec- 
tively. The filled diamond at z = 1.86 and the open circ le at 
z = 1.9 are t h e mea surements of lJanknecht et al.l l|2006l ) and 
iTvtler et al.l l|2004l ). respectively. Note that our values are 
in goo d agreement with those measured bv lKirkman et al.l 
(2005') at 1.7 < z < 3.3, although the latter did not include 
any LLSs and removed the metal contribution statistically. 

The solid line in Fig.[T3]is the best power-law fit to the 
optical depth values of high-resolution data in the redshift 
range 1.7 < z < 4 both from the literature as shown in 
the figure and our sample presented here. The errors in the 
Tg, measurements are taken into account in the fit. Despite 
the different method of measuring r^ , , we also included the 
Ki rkman et al. measur ement. Note that the point at z = 1.9 
by iTvtler et al.l (|2004l ) is from low resolution data and is 
therefore not includ ed in the fit. We did not include the 
ISchave et al. I (|2003l ) measurements from the spectra which 
are already included in our sample. The fit is given by r^, = 
(0.0023 ± 0.0007)(1 + 2)3e5±o.2i p^^. ^^^ sample alone the 
power law is t^, = (0.0054±0.0101)(l-h2)^'^'^*°'®^ This fit 
is shallower than that of the combined Th, due to a small 
number of QSOs and a lack of high-z QSOs in our sample 
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(cf. Kim et al. 2001; Kim et al. 2002) The dashed line is 
an extrapolated fit by iBernardi et al.1 l|2003h to their low 
resolution, low signal-to-noise SDSS data (their S/N > 4 
sample) in the redshilt ra nge 2.5 < z < 4, while the dot-dot- 
dot-dashed line is a fit bv lFan et al.1 (|2006r ) to measurements 
at 3 < 2 < 5.5 extrapolated to lower redshift, r^, oc (1-1- 
^-j4.3±o.3 rpjjg slope of the power law evolution measured by 
Bernard! et al. is in good agreement within la. As discussed 
extensively in the literature (e.g. Seljak et al. 2003; Viel 
et al. 2004b) there is, however, a systematic offset of about 
~ 25-30% between the Bernardi et al. r^ , and our and other 
measurements from high-resolution, high S/N data. This is 
most likely attributable to the difficulty of continuum fitting 
for low resolution, low signal-to-noise data which appears 
to lead to a systematic overestimate of the effective optical 
depth. The Fan et al. measurement (the dot-dot-dot-dashed 
line) has a somewhat steeper redshift evolution. This might 
indicate that the re is a deviation from a single po wer law at 
z > A. Note that iBecker. Ranch fc Sargent! (|2007l ) have also 
argued that the redshift evolution at 2; > 4 steepens towards 
higher redshift and is not well fit by a power law anymore. 
At 1.7 < z < 4, we find no evidence for a deviation from a 
power law. Table IA5I in Appendix A lists the measurement 



of T^j sampled at Az — 0.2 as well as r^, of each QSO 
belonging to each redshift bin. 



6 CONCLUSIONS 

We have obtained improved measurements of the flux prob- 
ability distribution at 1.7 < z < 3.2 and effective optical 
depth due to the intergalactic Hi absorption at 1.7 < z < 4, 
based on a sample of 18 high resolution, high signal-to-noise 
VLT/UVES QSO absorption spectra for which we have per- 
formed an extensive Voigt profile analysis of the Hi and 
metal absorption. 

The main results are as follows: 

(i) The normalised flux probability distribution (PDF) is 
affected mainly by metals at the level of 10-20% at ffux levels 
of 0.2 < F < 0.8 depending on redshift, and by continuum 
fltting uncertainties at the level of 5-20% at flux levels of 
0.8 < F < 1, depending on the signal-to-noise of the spectra. 
The effects of pixel noise and zero-level offset are very small, 
only noticeable at _F ~ and F ~ 1. The metal contribution 
to the absorption varies from a few percent to up to 30 
percent between different lines of sight. A careful individual 
removal of the metal absorption is therefore essential for an 
accurate determination of the shape of the PDF. 

(ii) Our new measurements of the flux PDF due to H I 
alone are systematically lower at 0.2 < F < 0. 7 than 
the widely-used PDFs measured bv lMcDonald et al.1 (|2000l ) 
based on a sample of Keck/HIRES spectra half the size 
of our sample. The difference has the same sign as would 
be expected if t he re moval of the metal absorption by 
iMcDonald et al.l (|2000l ) was less complete than ours, but 
it is actually somewhat larger and does not have the cor- 
rect redshift dependence if this were the sole cause. Given 
the rather small sample size cosmic variance is likely to be 
responsible for part of the discrepancy. 

(iii) The effect of our improved removal of metal ab- 
sorption on the measured effective optical depth due to 
Hi absorption, compared to previous measurements where 
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Figure 13. Comparison of the evolution of the effective optical 
depth of our full sample after removal of the metal absorption 
(filled circles) with that measured in Schaye et al. (2003, grey 
open squares) . There is a large overlap in the two samples and as 
expected the estimates from the two studies are in good agree- 
ment (for J2233— 606, see text). The horizontal and vertical error 
bars show the redshift range used and the \a errors. The er- 
rors were estimated using 500 bootstrap realisations of chunks of 
spectra with 100 pixels. The lower panel shows a comparison of 
the evolution of the effective optical depth of our sample divided 
into redshift bins with width Az = 0.2 (filled circles with errors) 
to a compilation of other measurements from the literature. The 
open squares are the measurements of Schaye et al. (2003) as in 
the upper panel. The filled squares, the filled triangles, the filled 
diamond at 2 = 1.86, and the open circle at z = 1.9 are from 
Kirkman et al. (2005), McDonald et al. (2000), Janknecht et al. 
(2006) and Tytler et al. (2004), respectively. The solid line is the 
best fit to all high-resolution data in the figure both from the lit- 
erature and the current sample in the redshift range 1.7 < z < 4: 
THi = (0.0023±0.0007)(l-|-2)3-'55±0.2i^ Tj^g dashed curve is a fit 

from Bernardi et al. (2003) to their S/N > 4 SDSS data in the 
redshift range from 2.5 < 2 < 4 extrapolated to lower redshift. 
The dot-dot-dot-dashed line is a fit by Fan et al. (2006) to data 
in the redshift range 3 < z < 5.5 extrapolated to lower redshift. 



the metal absorption has been taken into account in sim- 
pler ways, is small. Our new measurements of r^, are in 
good agreement with other measurements from high res- 
olution, high signal-to-noise spectra. In the redshift range 
1.7 < 2 < 4 the redshift evolution of the of our measure- 
ments of the effective optical depth and other measurements 
from high resolution spectra is best fit with a single power 
law rS« = (0.0023 ± 0.0007) (1 -H 2)3-65±o.2i^ 



ACKNOWLEDGMENTS. 

We would like to thank ESO, the ESO staff, the ESO science 
verification team and the UVES LP team of "The cosmic 
evolution of the intergalactic medium" for initiating, com- 
piling and making publicly available a superb set of QSO ab- 
sorption spectra. We also thank the UVES team for building 



© 0000 RAS, MNRAS 000, 000-000 



1^ 



Kim et al. 



the spectrograph and our referee Michael Strauss for his use- 
ful suggestions. TSK would like to thank Michael Murphy 
for providing his spectral combining program, UVES_popler, 
and the loA, Cambridge, UK, for hospitality during the final 
stages of this work. 



McDonald P., Miralda-Escude J., Ranch M., Sargent W.L., 
Barlow T.A., Cen R., Ostriker J.R, 2000, ApJ, 543, 1 

McDonald P., Seljak, U., Buries S. et al., 2006, ApJS, 163, 
80 

Meiksin A., Bryan G. & Machacek M., 2001, MNRAS, 327, 
296 



REFERENCES 

Becker G., Ranch M., Sargent W. L. W., 2007, ApJ, 662, 

72 
Bergeron J., Petitjean P., Aracil B. et al., 2004, Msngr, 118, 

40 
Bernard! M., Sheth R. K., SubbaRao M. et al., 2003, AJ, 

125, 32 
Bolton J. S., Haehnelt M. G., Viel M., Springel V., 2005, 

MNRAS, 357, 1178 
Bryan G. L., Machacek M. E., Anninos P., Norman M. L., 

1999, ApJ, 517, 13 
Carswell R.F., Schaye J., Kim T.-S., 2002, ApJ, 578, 43 
Cen R. Miralda-Escude J., Ostriker J. P., Ranch M., 1994, 

ApJ, 437, L9 
Cowie L. L., Songaila A., Kim T.-S., Hu E. M., 1995, AJ, 

109, 1522 
Cristiani S., D'Odorico V., 2000, AJ, 120, 1648 
Croft R. A. C, Weinberg D. H., Katz N., Hernquist L., 

1998, ApJ, 495, 44 
Croft R.A.C., Weinberg D. H., Bolte M., Buries S., Hern- 
quist L., Katz N.,Kirkman D., Tytler D., 2002, ApJ, 581, 

20 
Dave R., Hernquist L., Katz N., Weinberg D. H., 1999, 

ApJ, 511, 521 
Dekker H., D'Odorico S., Kaufer A., Delabre B., Kot- 

zlowski H., 2000, Proc. SPIE, 4008, 534 
Desjacques V. & Nusser A., 2005, MNRAS, 361, 1257 
Desjacques V., Nusser A., Sheth R. K., 2007, MNRAS, 374, 

206 
Fan X., Strauss M. A., Becker R. H. et al., 2006, AJ, 132, 

117 
Fechner C, Baade R, Reimers D., 2004, A&A, 418, 857 
Hu E. M., Kim T.-S., Cowie L. L., Songaila A., Ranch M., 

1995, AJ, 110, 1526 

Janknecht E., Reimers D., Lopez S., Tytler D., 2006, A&A, 

458, 427 
Jena T., Norman M. L., Tytler D. et al., 2005, MNRAS, 

361, 70 
Jenkins E. B. & Ostriker J. P., 1991, ApJ, 376, 33 
Kim T.-S., Cristiani S., D'Dodorico S., 2001, A&A, 373, 

757 
Kim T.-S., Carswell R. F., Cristiani S., D'Odorico S., Gi- 

allongo E., 2002, MNRAS, 335, 555 
Kim T.-S., Viel M., Haehnelt M. G., Carswell R. F., Cris- 
tiani S., 2004, MNRAS, 347, 355 
Kirkman D., Tytler D., 1997, ApJ, 484, 672 
Kirkman D., Tytler D., Suzuki, N. et al., 2005, MNRAS, 

360, 1373 
Lesgourgues J., Viel M., Haehnelt M.G., Massey R., 2007, 

submitted, astroph/07050533 
Lidz A., Heitmann K., Hui L., Habib S., Ranch M., Sargent 

W. L. W., 2006, ApJ, 638, 27 
Lu L., Sargent W. L. W., Womble D. S., Takada-Hidai M., 

1996, ApJ, 472, 509 



ApJ, 471, 58 
Morton D. C, 2003, ApJS, 149, 205 
Press W. H., Rybicki G. B., Schneider D. P., 1993, ApJ, 

414, 64 
Ranch M., Miralda-Escude J., Sargent W. L. W., Barlow 

T. A., Weinberg D. H., Hernquist L., Katz N., Cen R., 

Ostriker J. P., 1997, ApJ, 489, 7 
Schachter J., 1991, PASP, 103, 457 
Schaye J., Aguirre A., Kim T.-S., Theuns T., Rauch R., 

Sargent W. L. W., 2003, ApJ, 301, 478 
Seljak U., McDonald P., & Makarov A., 2003, MNRAS, 

342, L79 
Songaila A., 2004, AJ, 127, 2598 
Theuns T., Leonard A., Efstathiou G., Pearce F. R., 

Thomas P. A., 1998, MNRAS, 301, 478 
Theuns T., Schaye J., Haehnelt M. G., 2000, MNRAS, 215, 

600 
Tytler D. & Fan X.-M., 1992, ApJS, 79, 1 
Tytler D., Kirkman D., O'Meara J. M. et al., 2004, ApJ, 

617, 1 



Viel M., Haehnelt M. G., Carswell R.F., Kim T.-S., 2004a, 

MNRAS, 349, L33 
Viel M., Haehnelt M. G., Springer V., 2004b, MNRAS, 354, 

684 
Viel M., Haehnelt M. G. & Lewis A., 2006, MNRAS, 370, 

51 
Weinberg D. H., Buries S. Croft R. A. C. et al., 1998, in 

Evolution of Large Scale Structure: From Recombination 

to Garching, eds. A. J. Banday, R. K. Sheth & L. N. Da 

Costa (Twin Press: Vledder NL), p. 346 
Zhang Y., Anninos P., Norman M. L., Meiksin, A., 1997, 

ApJ, 485, 496 



© 0000 RAS, MNRAS 000, 000-000 



The flux distribution of the Lya forest 19 



Table Al. The wavelength range of each spectrum in the three 
redshift bins 



QSO <z>=2.07 <z> = 2.52 <z>=2.94 

Q0055-269 4785-5112 

PKS2126-158 4638-5112 

Q0420-388 4231-4506 4506-4909 

HE0940-1050 4197-4502 4502-4870 

HE2347-4342 4055-4502 4502-4643 

Q0002-422 3901-4092 4092-4504 

PKS0329-255 3815-4090 4090-4439 

Q0453-423 4084-4362 

HE1347-2457 3705-4091 4091-4319 

Q0329-385 3528-4105 

HE2217-2818 3509-4091 

Q0109-3518 3532-4070 

HE1122-1648 3514-4082 

J2233-606 3335-3886 

PKS0237-23 3361-3865 

PKS1448-232 3306-3860 

Q0122-380 3306-3819 

QllOl-264 3503-3765 



APPENDIX A: TABLES 
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Table A2. Uncertain line fits for individual QSOs'' 



QSO 



Uncertain line fits 



Q0055-269 Sim at ^ = 3.2562 (m), Sin 1260 a.t z = 3.2562 (1) 

PKS2126-158 Sin 1260 at ^ = 2.9071 (1), Sim at z = 2.9067 (m), Sim at 2 = 2.7668 (m) 

Q0420-388 Sim 1260 at 2 = 2.5372 (1), Siiv 1394 at 2 = 2.2475 (1), Siiv 1403 at 2 = 2.2475 (1) 

HE0940-1050 Sin 1260 at 2 = 2.8346 (1), Cii 1334 at 2 = 2.3306 (1), Oi 1302 at 2 = 2.3292 (1), Cii 1334 at 2 = 2.2212 (m) 

HE2347-4342 Sim at 2 = 2.7412 (m), Sin 1260 at 2 = 2.3124 (1) 

Q0002-422 Sin 1260 at 2 = 2.4623 (1), Sim at 2 = 2.4642 (m), Sim at 2 = 2.3022 (m) 

PKS0329-255 Sim at 2 = 2.3289 (m) 

Q0453-423 Aln at 2 = 1.6172 (1) 

HE1347-2457 Civ 1548 at 2 = 1.6083 (m) 

Q0329-385 

HE2217-2818 Sim at 2 = 1.9653 (1), Cn 1334 at 2 = 1.690983 (1) 

Q0109-3518 Cn 1334 at 2 = 2.0463 (m), Sim at 2 = 2.0473 (1) 

HE1122-1648 Sim at 2 = 2.0071 (m) 

J2233-606 Sim at 2 = 1.9426 (h), Sim at 2 = 1.9257 (m) Sim at 2 = 1.8711 (m), Cn 1334 at 2 = 1.8711 (1) 

PKS0237-23 Civ 1548, 1550 at 1.3643 (m), Sim at 2 = 2.1781, Cn 1334 at 2 = 1.6756 (m), Cii 1334 at 2 = 1.6359 (m) 

PKS1448-232 Cn 1334 at 2 = 1.5855 (1) 

Q0122-380 Sim at 2 = 1.9733 (m). Sin 1260 at 2 = 1.9695 (1), Sim at 2 = 1.9123 (1), Sim at 2 = 1.9102 (m) 

QllOl-264 

^ The letters "m" and "1" indicate a "moderate" uncertainty and a "low" uncertainty, respectively. For the class "1" the H I profiles do 
not change significantly with and without the uncertain metal lines (c/. Sin 1260 in Fig. |3]f). The example of the Sim absorption in 
Fig- Elf) is a typical uncertain line fit of class "m". 

Table A3. The mean PDF of the full sample divided into three and two redshift bins. 



F 


< 2>=2.07 


<2>=2.52 


< 2>=2.94 


< 2 >= 2.41" 


< 2 > = 3.00" 


0.00 


0.5700± 0.0581 


0.9189± 0.0995 


1.6960± 0.2043 


0.8009± 0.1114 


1.6277± 0.2356 


0.05 


0.1978± 0.0165 


0.3540± 0.0320 


0.4666± 0.0520 


0.2870± 0.0362 


0.4861± 0.0603 


0.10 


0.1354± 0.0115 


0.2426± 0.0190 


0.2957± 0.0312 


0.1851± 0.0227 


0.3210± 0.0396 


0.15 


0.1401± 0.0098 


0.2064± 0.0199 


0.2799± 0.0281 


0.1664± 0.0215 


0.2950± 0.0371 


0.20 


0.1230± 0.0083 


0.1873± 0.0151 


0.2356± 0.0230 


0.1624± 0.0172 


0.2574± 0.0330 


0.25 


0.1307± 0.0078 


0.2230± 0.0206 


0.2403± 0.0212 


0.1822± 0.0177 


0.2412± 0.0291 


0.30 


0.1216± 0.0086 


0.2003± 0.0168 


0.2385± 0.0197 


0.1630± 0.0141 


0.2350± 0.0230 


0.35 


0.1279± 0.0083 


0.1995± 0.0151 


0.2310± 0.0198 


0.1517± 0.0135 


0.2323± 0.0273 


0.40 


0.1450± 0.0088 


0.2151± 0.0133 


0.2502± 0.0195 


0.1941± 0.0146 


0.2646± 0.0283 


0.45 


0.1488± 0.0083 


0.2574± 0.0196 


0.2496± 0.0166 


0.1907± 0.0147 


0.2448± 0.0188 


0.50 


0.1726± 0.0099 


0.2561± 0.0180 


0.2817± 0.0195 


0.2162± 0.0198 


0.2717± 0.0265 


0.55 


0.1884± 0.0104 


0.2800± 0.0170 


0.3523± 0.0201 


0.2462± 0.0207 


0.3480± 0.0235 


0.60 


0.2209± 0.0114 


0.3323± 0.0206 


0.4246± 0.0228 


0.3158± 0.0285 


0.4009± 0.0247 


0.65 


0.2815± 0.0144 


0.3588± 0.0195 


0.4555± 0.0351 


0.3702± 0.0306 


0.4269± 0.0297 


0.70 


0.3067± 0.0136 


0.4107± 0.0183 


0.5785± 0.0342 


0.4154± 0.0254 


0.5793± 0.0432 


0.75 


0.4045± 0.0170 


0.5631± 0.0284 


0.6888± 0.0485 


0.5417± 0.0362 


0.7354± 0.0582 


0.80 


0.5747± 0.0239 


0.8030± 0.0358 


0.9168± 0.0587 


0.7511± 0.0513 


0.9470± 0.0713 


0.85 


0.8440± 0.0317 


1.1532± 0.0712 


1.4003± 0.0660 


1.0816± 0.0731 


1.4178± 0.0733 


0.90 


1.6501± 0.0464 


2.0285± 0.1072 


2.1911± 0.0841 


2.0036± 0.1064 


2.1899± 0.0929 


0.95 


4.9083± 0.0982 


4.6174± 0.1175 


3.9641± 0.1729 


4.9196± 0.1604 


3.9674± 0.1805 


1.00 


8.6081± 0.1986 


6.1926± 0.3069 


4.5630± 0.3659 


6.6550± 0.3420 


4.5108± 0.4290 



'^ The wavelength ranges for < 2 > = 2.41 and < 2 > = 3.00 are 3902-4325 A (5 QSOs: HE2347-4342, Q0002-422, PKS0329-255, 
Q0453-423 and HE1347-2457), and 4703-5112 A (4 QSOs: Q0055-269, PKS2126-158, Q0420-388 and and HE0940-1050), respec- 
tively. 
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Table A4. The Hi effective optical depth'' 



QSO 


wavelengths (A) 


<z> 




Fhi 


-% 


Hl+motal 


Metals (%) 


Q0055-269 


4785-5577 


3.262 


0.412±0.029 


0.662±0.020 


0.1233±0.0065 


0.419±0.030 


1.6 




4785-5112 


3.071 


0.386±0.046 


0.680±0.033 


0.1296±0.0119 


0.388±0.046 


0.5 


PKS2126-158 


4638-5112 


3.010 


0.263±0.025 


0.769±0.019 


0.0949±0.0086 


0.351±0.031 


25.0 


Q0420-388 


4231-4909 


2.759 


0.308±0.024 


0.735±0.017 


0.1092±0.0068 


0.332±0.025 


7.2 


HE0940-1050 


4197-4870 


2.729 


0.283±0.023 


0.754±0.018 


0.1004±0.0086 


0.325±0.025 


13.0 


HE2347-4342 


4055-4643 


2.577 


0.236±0.022 


0.790±0.018 


0.0900±0.0092 


0.241±0.022 


2.3 


Q0002-422 


3901-4504 


2.457 


0.250±0.021 


0.778±0.016 


0.0912±0.0081 


0.325±0.025 


23.1 


PKS0329-255 


3815-4439 


2.395 


0.183±0.020 


0.833±0.015 


0.0762±0.0088 


0.196±0.020 


6.4 


Q0453-423 


4084-4362 


2.474 


0.253±0.031 


0.777±0.024 


0.0904±0.0113 


0.271±0.033 


6.7 




3758-3911 


2.154 


0.147±0.026 


0.864±0.022 


0.0523±0.0110 


0.161±0.025 


9.0 


HE1347-2457 


3705-4319 


2.300 


0.140±0.015 


0.870±0.013 


0.0544±0.0079 


0.165±0.016 


15.4 


Q0329-385 


3528-4105 


2.139 


0.146±0.013 


0.864±0.012 


0.0543±0.0066 


0.150±0.013 


2.7 


HE2217-2818 


3509-4091 


2.126 


0.138±0.014 


0.871±0.014 


0.0557±0.0076 


0.168±0.016 


17.8 


Q0109-3518 


3532-4070 


2.127 


0.143±0.015 


0.867±0.014 


0.0590±0.0078 


0.168±0.016 


14.7 


HE1122-1648 


3514-4082 


2.124 


0.148±0.017 


0.863±0.014 


0.0603±0.0079 


0.161±0.017 


8.0 


J2233-606 


3350-3886 


1.976 


0.164±0.019 


0.849±0.017 


0.0707±0.0106 


0.179±0.019 


8.4 


PKS0237-23 


3361-3865 


1.972 


0.127±0.015 


0.880±0.013 


0.0527±0.0077 


0.176±0.019 


27.6 


PKS1448-232 


3306-3860 


1.947 


0.145±0.014 


0.865±0.013 


0.0604±0.0071 


0.157±0.014 


7.6 


Q0122-380 


3282-3819 


1.921 


0.133±0.018 


0.875±0.015 


0.0613±0.0096 


0.160±0.019 


16.4 


QllOl-264 


3503-3765 


1.989 


0.111±0.021 


0.895±0.018 


0.0435±0.0098 


0.129±0.022 


14.1 




3233-3398 


1.727 


0.096±0.020 


0.908±0.018 


0.0316±0.0110 


0.131±0.026 


26.9 



^ The errors were estimated using 500 bootstrap realisations of chunks of 100 pixels (5 A). 

Table A5. The evolution of the effective optical depth of the full sample divided into redshift bins with width A2=0.2'' 



<z > 



Wavelengths (A) 






<z> 



Wavelengths (A) 






1.80 



2.00 



2.20 







3282-3525 


0.127±0.013 


J2233- 


■606 


3335-3525 


0.203±0.039 


PKS02: 


37-23 


3361-3525 


0.111±0.024 


PKS14 


48-232 


3306-3525 


0.126±0.022 


Q0122- 


-380 


3282-3525 


0.097±0.018 


QllOl- 


-264 


3282-3398 


0.095±0.027 






3525-3769 


0.141±0.008 


Q0329- 


-385 


3528-3769 


0.128±0.017 


HE221' 


?-2818 


3525-3769 


0.133±0.023 


Q0109- 


-3518 


3532-3769 


0.156±0.026 


HE1122-1648 


3525-3769 


0.117±0.019 


J2233- 


■606 


3525-3769 


0.168±0.039 


PKS02; 


37-23 


3525-3769 


0.146±0.022 


PKS14 


48-232 


3525-3769 


0.157±0.021 


Q0122- 


-380 


3525-3769 


0.165±0.017 


QllOl- 


-264 


3525-3765 


0.098±0.021 






3769-4012 


0.147±0.009 


Q0002- 


-422 


3901-4012 


0.269±0.046 


PKS03: 


29-255 


3815-4012 


0.191±0.036 


Q0453- 


-423 


3769-3911 


0.142±0.022 


HE134' 


r-2457 


3769-4012 


0.094±0.021 


Q0329- 


-385 


3769-4012 


0.138±0.017 


HE221' 


r-2818 


3769-4012 


0.152±0.022 


Q0109- 


-3518 


3769-4012 


0.127±0.027 


HE1122-1648 


3769-4012 


0.163±0.019 


J2233- 


606 


3769-3886 


0.096±0.019 



2.41 



2.59 



2.80 



2.99 



3.18 



3.44'' 



HE2347-4342 

Q0002-422 

PKS0329-255 

Q0453-423 

HE1347-2457 

Q0420-388 

HE0940-1050 

HE2347-4342 

Q0002-422 

PKS0329-255 

Q0453-423 

PKS2126-158 
Q0420-388 
HE0940-1050 
HE2347-4342 

Q0055-269 
PKS2126-158 
Q0420-388 
HE0940-1050 

Q0055-269 

PKS2126-158 

Q0055-269 



4012- 
4055- 
4012- 
4012- 
4084- 
4012- 
4255- 
4255- 
4255- 
4255- 
4255- 
4255- 
4255- 
4498- 
4638- 
4498- 
4498- 
4498- 
4741- 
4785- 
4741- 
4741- 
4741- 
4984- 
4984- 
4984- 
5227- 



4255 
4255 
4255 
4255 
4255 
4255 
4498 
4498 
4498 
4498 
4498 
4439 
4362 
4741 
4741 
4741 
4741 
4643 
4984 
4984 
4984 
4914 
4870 
5227 
5227 
5112 
-5577 



0.204±0.015 
0.156±0.022 
0.228±0.034 
0.186±0.034 
0.252±0.042 
0.205±0.022 
0.255±0.016 
0.266±0.031 
0.293±0.033 
0.258±0.022 
0.268±0.034 
0.170±0.037 
0.255±0.060 
0.297±0.024 
0.231±0.052 
0.308±0.034 
0.304±0.033 
0.315±0.025 
0.338±0.024 
0.389±0.062 
0.310±0.044 
0.380±0.038 
0.260±0.046 
0.304±0.035 
0.362±0.050 
0.203±0.059 
0.462±0.045 



-off 



^ The errors were estimated using 500 bootstrap realisations of chunks of 100 pixels (5 A). For each bin, rS'^ was estimated from the 
mean flux of all pixels from the QSOs listed. 
^ Q0055-269 is the only QSO in this z bin. 
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